Apparatus and method for removing red-eye in a two-dimensional (2d) image

ABSTRACT

An apparatus and method for removing red-eye in a 2D image, which can remove red-eye more naturally and apply a gain for a real-time correction on input pixels by differently performing a correction degree according to positions of pixels included in red-eye.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a U.S. divisional application filed under 37 CFR1.53(b) claiming priority benefit of U.S. Ser. No. 12/273,878, filed inthe United States on Nov. 19, 2008, now allowed, which claims earlierpriority benefit to Korean Patent Application No. 2008-21236 filed onMar. 7, 2008, and the priority of Korean Patent Application No.2008-44444 filed on May 14, 2008, the disclosures of which areincorporated herein by reference.

BACKGROUND

1. Field

The present invention relates to a digital image processing field, andin particular, to a Two-Dimensional (2D) image segmentation apparatusand method, which segment the pixels of a progressive input image fordetecting a specific pattern of the image, and an apparatus and methodfor removing red-eye in 2D image, which can remove red-eye in thesegmented area.

2. Description of the Related Art

Recently, a pattern recognition technology that recognizes a specificshape through an image process is applied to a variety of industryfields. Examples of the pattern recognizing technology includewell-known pattern recognition technologies such as a patternrecognition system that detects eyelid tremor in order to prevent todrive while drowsy, a character recognition system that recognizescharacters from an image scanned through a scanner, and a fingerprintrecognition system.

Such a pattern recognition technology requires a technology thatsegments the pixels of an entire image in order to detect shapes forrecognition from the entire image generated by a camera and the like.

Generally, a segmentation algorithm used in the pattern recognitiontechnology includes the first analysis process that analyzes a featureof a portion of an input image to be detected from the input image, andthe second analysis process that transforms the input image into abinary image representing only specific information and thereafterrecognizes it as one group using the shape information of a portion ofthe input image to be detected through the first analysis process. Forexample, the pattern recognition process, which recognizes eye regionsfor removing red-eye in an image photographed by a camera, is a processthat generates the binary image representing only pixels representingred in an image photographed through the first analysis process, andsegments each of pixels of the binary image through the second analysisprocess to determine groups corresponding to the shapes of eyes. In sucha pattern recognition process, the segmentation algorithm may beperformed.

Examples of a related art image segmentation algorithm include agrass-fire algorithm that detects patterns in eight directions at onepoint to perform segmentation, and an algorithm that pre-inputsinformation of patterns and detects patterns using a correlation with aninput image. However, in a case of the related segmentation algorithm,since an operation time is very slow and a complicated operation isrequired, a microprocessor must necessarily be used for the operationprocess. Accordingly, in a case of the related art segmentationalgorithm, it is impossible to perform a real-time segmentation processon the pixels of a progressive input image, and the cost increases dueto an expensive microprocessor. Moreover, it is difficult to apply therelated art segmentation algorithm to devices requiring miniaturizationsuch as portable phones.

On the other hand, the related art image segmentation algorithm can beapplied to a technology for removing red-eye in a digital image. Thatis, a segmentation on red-eye occurring areas in a Two-Dimensional (2D)digital image is performed using the related art segmentation algorithm,and a red-eye removing algorithm can be applied using the coordinates ofthe segmented areas.

Generally, examples of the related art technology for removing red-eyeinclude a scheme that replaces a red-eye occurring area with a specificcolor, and a scheme that manipulates an image through masking.

The scheme replacing with a specific color that determines which area aninput pixel is in, and thereafter checks whether a color of acorresponding pixel is in a skin color area or a color corresponding tored-eye when the input pixel is in an area to be manipulated. At thispoint, when the check result shows that the color of the correspondingpixel is the color corresponding to red-eye, the corresponding pixel isreplaced with black. Since such a scheme requires a small amount ofoperation and a small memory space, it requires a very fast operationspeed and a small memory space. However, when a pixel determined asred-eye is replaced with a specific color, an image-manipulated fact canclearly be represented because the colors of eyes become monotonous.

The masking-based scheme is a scheme that makes a color near to black byapplying a mask having the same size as that of an area having a lowcoefficient to the area to be manipulated. Because the masking-basedscheme can make the colors of eyes natural, it reduces the possibilitythat users can recognize a color-manipulated fact. However, themasking-based scheme requires a space to store an area to bemanipulated, and thus a size of a memory space increases becauseinformation to be stored increases as a size of an area to bemanipulated increases. Furthermore, in a case of the masking-basedscheme, since a size of an area to be manipulated varies, the mask mustalso be variable. Moreover, since an operation speed becomes slow as anoperation area increases, the masking-based scheme is unsuitable for asystem requiring a high-speed operation.

SUMMARY

An aspect of the present invention provides a Two-Dimensional (2D) imagesegmentation apparatus and method, which can perform a real-timesegmentation on each of the pixels of a progressive input 2D image andrequire no additional hardware resources due to the simplification of anoperation.

Another aspect of the present invention provides an apparatus and methodfor removing red-eye in 2D image, which can remove red-eye morenaturally and apply a gain for a real-time correction on input pixels bydifferently performing a correction degree according to positions ofpixels included in red-eye.

According to an aspect of the present invention, there is provided a 2Dimage segmentation apparatus for segmenting pixels of a progressiveinput 2D image, including: a group information storing unit storinginformation of pixel groups including a plurality of adjacent pixels; apixel determining unit determining coordinates of an input pixel, anddetermining whether the input pixel is an effective pixel forsegmentation; a group scanning unit scanning a adjacent pixel groupdisposed in a scan range preset from the effective pixel in the groupstoring unit when the input pixel is determined as the effective pixelby the pixel determining unit; and a group information updating unitupdating information of a pixel group stored in the group informationstoring unit according to whether there is the scanned adjacent pixelgroup and a pixel group including an input pixel preceding the effectivepixel.

According to another aspect of the present invention, there is provideda 2D image segmentation method for segmenting pixels of a progressiveinput 2D image, including: determining coordinates of an input pixel,and determining whether the input pixel is an effective pixel forsegmentation; scanning a adjacent pixel group disposed in a scan rangepreset from the effective pixel in the group storing unit storinginformation of a pixel group when the input pixel is determined as theeffective pixel; and updating information of the pixel group stored inthe group information storing unit according to whether there is thescanned adjacent pixel group and a pixel group including an input pixelpreceding the effective pixel.

According to another aspect of the present invention, there is providedan apparatus for removing red-eye in 2D image, including: a referencelength calculating unit receiving coordinate information of an areaincluding red-eye in an entire image, and determining center coordinatesof the area and a reference length from the center coordinates; a pixellength calculating unit sequentially receiving each of pixels of theimage, and calculating a pixel length being a distance between the eachpixel and the center of the area; a section determining unit comparingthe reference length with a pixel length, selecting a pixel, the pixellength of which is less than the reference length, and determining asection including the selected pixel among predetermined sectionsaccording to a distance from the center coordinates; and an imagemanipulating unit setting a brightness gain and color gain for eachsection, and applying the brightness gain and the color gain to correcta pixel value of the selected pixel.

According to another aspect of the present invention, there is provideda method for removing red-eye in 2D image, including: receivingcoordinate information of an area including red-eye in an entire image;determining a reference length from the center coordinates and centercoordinates of the received area; sequentially receiving each of pixelsof the image, and calculating a pixel length being a distance betweenthe each pixel and the center of the area; comparing the referencelength with a pixel length, selecting a pixel, the pixel length of whichis less than the reference length, and determining a section includingthe selected pixel among predetermined sections according to a distancefrom the center coordinates; and setting a brightness gain and colorgain for each section, and applying the brightness gain and the colorgain to correct a pixel value of the selected pixel.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and other advantages of thepresent invention will be more clearly understood from the followingdetailed description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a block diagram of a Two-Dimensional (2D) image segmentationapparatus according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a 2D image segmentation methodaccording to an embodiment of the present invention;

FIGS. 3A and 3B is an exemplary diagram for describing a binary image towhich an embodiment of the present invention is applied;

FIG. 4 is an exemplary diagram of one pixel group which is segmented;

FIG. 5 illustrates a plurality of registers storing group information ofthirty-two groups;

FIGS. 6A, 6B, 7A, 7B, 8A and 8B are exemplary diagrams for describing amethod for setting an adjacent pixel group scan range according to anembodiment of the present invention;

FIG. 9 is a flowchart illustrating a method for updating groupinformation pre-stored in the group information storing unit 11 when anadjacent pixel group is scanned according to an embodiment of thepresent invention;

FIG. 10 is an exemplary diagram illustrating types of a plurality ofpixel groups stored in a group information storing unit according to anembodiment of the present invention;

FIG. 11 is a block diagram of an apparatus for removing red-eye in 2Dimage according to an embodiment of the present invention;

FIG. 12 is a block diagram of a method for removing red-eye in 2D imageaccording to an embodiment of the present invention;

FIG. 13 is a flowchart illustrating in more detail an operation ofcorrecting a pixel value of FIG. 12;

FIG. 14 is a conceptual view for describing the calculation of thecoordinates, reference length and coordinate length of an applied areain an embodiment of the present invention;

FIG. 15 is a graph illustrating a change rate when correction isperformed on a selected pixel according to a relationship between areference length and a coordinate length in an embodiment of the presentinvention;

FIGS. 16 and 17 are graphs illustrating a change of a brightness gainaccording to a position of a selected pixel in an embodiment of thepresent invention;

FIGS. 18 and 19 are graphs illustrating a change of a color gainaccording to a position of a selected pixel in an embodiment of thepresent invention;

FIGS. 20 and 21 are graphs illustrating a change of a brightness gainapplied to a highlight process on a section including center coordinatesin an embodiment of the present invention; and

FIG. 22 is an exemplary diagram of an eye image to which a highlightprocess applied on a section including center coordinates in anembodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

Exemplary embodiments of the present invention will be described belowin more detail with reference to the accompanying drawings. The presentinvention may, however, be embodied in different forms and should not beconstrued as limited to the embodiments set forth herein. Rather, theseembodiments are provided so that this disclosure will be thorough andcomplete, and will fully convey the scope of the present invention tothose skilled in the art. Accordingly, it is noted that the dimensionsof the elements illustrated in the accompanying drawings may beexaggerated for clarity.

FIG. 1 is a block diagram of a Two-Dimensional (2D) image segmentationapparatus according to an embodiment of the present invention.

Referring to FIG. 1, the 2D image segmentation apparatus 10 according toan embodiment of the present invention includes a group informationstoring unit 11 storing information of pixel groups consisting of aplurality of adjacent pixels, a pixel determining unit 12 determiningthe coordinates of a pixel of a progressive input 2D image, anddetermining whether the input pixel is an effective pixel forsegmentation, a group scanning unit 13 scanning a adjacent pixel groupdisposed in a scan range preset from the effective pixel in the groupstoring unit 11 when the input pixel is determined as the effectivepixel by the pixel determining unit 12, and a group information updatingunit 14 updating information of a pixel group stored in the groupinformation storing unit 11 according to whether there is the scannedadjacent pixel group and a pixel group including an input pixelpreceding the effective pixel.

The 2D image segmentation apparatus 10 according to an embodiment of thepresent invention may further include a group feature determining unit15 determining whether a corresponding pixel group is effectiveaccording to the number of pixels included in a pixel group determinedas the update completion of group information among the pixel groupsstored in the group information storing unit 11 and the width-to-heightratio of the pixel group determined as the update completion of thegroup information, and deleting group information of a pixel groupdetermined as an ineffective pixel group from the group informationstoring unit 11.

FIG. 2 is a flowchart illustrating a 2D image segmentation methodaccording to an embodiment of the present invention.

Referring to FIG. 2, the 2D image segmentation method according to anembodiment of the present invention includes operations S11 to S17. Theoperation S11 determines the coordinates of a pixel of a progressiveinput 2D image, and determines whether the input pixel is effectivepixel for segmentation. When the input pixel is determined as anineffective pixel for segmentation by the pixel determining unit 12 inthe operation S12, the operation S13 is in a standby state for receivinga succeeding input pixel. When the input pixel is determined as theeffective pixel for segmentation by the pixel determining unit 12 in theoperation S12, the operation S14 scans a adjacent pixel group disposedin a scan range preset from the effective pixel in the group informationstoring unit 11 storing information of pixel groups. The operations S15to S17 updates information of a pixel group stored in the groupinformation storing unit 11 according to whether there exists thescanned adjacent pixel group and a pixel group including an input pixelpreceding the effective pixel.

The 2D image segmentation method according to an embodiment of thepresent invention may further includes an operation S18 that determineswhether a corresponding pixel group is effective according to the numberof pixels included in a pixel group determined as the update completionof group information among the pixel groups stored in the groupinformation storing unit 11 and the width-to-height ratio of the pixelgroup determined as the update completion of the group information, andan operation (not shown) that deletes group information of a pixel groupdetermined as an ineffective pixel group from the group informationstoring unit 11.

Hereinafter, the 2D image segmentation apparatus and method according toan embodiment of the present invention having the elements of FIGS. 1and 2 will be described in detail with reference to the accompanyingdrawings.

Generally, in a case of an image photographed by an image sensor(Charge-Coupled Device (CCD) or Complementary Metal-Oxide Semiconductor(CMOS)) such as a digital camera, all pixels included in one frame arenot input at a time but sequentially input according to a certain order.Particularly, pixels, which have a matrix structure and are disposed inan image, have a progressive input form that pixels included in onecolumn are input in a row order and pixels included in a succeedingcolumn are input in a row order, wherein the input of pixels starts froma pixel disposed in the first row of the first column. An embodiment ofthe present invention may be applied in an input order on each of imagepixels which are input in the progressive input form.

An input image to which an embodiment of present invention is appliedmay be a binary image which represents only a pixel having a desiredfeature by applying a specific mask to a common image. For example, in acase where pixels representing red are extracted for removing red-eyefrom a general image illustrated in FIG. 3( a), if a red mask isapplied, a binary image can be obtained in which only pixels includingred more than a certain level is displayed in black (have a value of 1)and another pixels are displayed in white (have a value of 0) asillustrated in FIG. 3( b).

In an embodiment of the present invention as described above, when abinary image representing pixels having specific information is input ina progressive-scanning scheme, the pixel determining unit 12 determinesthe coordinates of a current input pixel and determines whether theinput pixel is a pixel for segmentation in the operation S11. Forexample, the pixel determining unit 12 can determine whether the inputpixel has a value of 1 or 0 in the binary image and determine a pixelhaving the value of 1 as an effective pixel. A pixel that is notdetermined as the effective pixel by the pixel determining unit 12,i.e., a pixel having the value of 0 is not applied to a set of processesaccording to an embodiment of the present invention described below, andthe pixel determining unit 12 is in a standby state for receiving asucceeding input pixel in the operation S13.

The group scanning unit 13 scans a group adjacent to a current effectivepixel using information of the pixel groups pre-stored in the groupinformation storing unit 11, on pixels determined as the effective pixelby the pixel determining unit 12 in the operations S12 and S14.

The group information storing unit 11 is an element storing informationof the pixel groups consisting of a plurality of adjacent pixels, maystore the information of each of the pixel groups in a register form.FIG. 4 is an exemplary diagram of one pixel group which is segmented.FIG. 5 illustrates a plurality of registers storing group information ofthirty-two groups. As illustrated in FIG. 4, group information of onepixel group may be represented as the horizontal minimum coordinatex_min, horizontal maximum coordinate x_max, vertical minimum coordinatey_min and vertical maximum coordinate y_max included in the pixel groupand the number of pixels grp_pixel_count included in the pixel group.Moreover, as illustrated in FIG. 5, all group information may be storedin a register form.

As illustrated in FIG. 5, the group information storing unit 11 mayinclude a flag grp_Flag representing whether the group information canbe updated by a current input pixel. The flag grp_Flag representswhether a corresponding pixel group is in an activated state where thegroup information can be updated by the current input pixel. Forexample, in a case where the vertical maximum coordinate y_max of acorresponding pixel group is not within a vertical scan range of thecurrent input effective pixel, since information of the correspondingpixel group is no longer updated, a deactivation state can berepresented in the flag grp_Flag. An effectiveness determining processon a pixel group to be described below may be performed according to theflag grp_Flag.

In the operation S14 where the group scanning unit 13 scans an adjacentpixel group of an effective pixel, the group scanning unit 13 can scanan adjacent group in a preset scan range which is input from the outsideby a user. A process of scanning the adjacent group may largely bedivided into two schemes according to a shape of a scan range formedfrom a current effective pixel.

As illustrated in FIG. 6( a), a first scheme is a scheme that scansadjacent pixel groups by applying a preset horizontal scan rangex_scan_range having the same size to an entire pixel group (group 2,group 3), at least one portion of which is included in a vertical scanrange y_scan_range preset from an effective pixel P. Since such a firstscheme applies the horizontal scan range x_scan_range having the samesize to an entire pixel group, at least one portion of which is includedin the vertical scan range y_scan_range, it may very simply beimplemented. As a result, as illustrated in FIG. 6( b), the first schemeis the same as a scheme that represents a square shape of a window,which is represented as the horizontal scan range and the vertical scanrange in a current effective pixel, as a scan range. However, since thefirst scheme applies the same horizontal scan range irrespective of avertical distance, a pixel group, which exists in a diagonal of thesquare window which is more distant than a pixel group which is notincluded in the square window in a vertical or horizontal direction, mayinclude an adjacent pixel group.

As illustrated in FIG. 7( a) or FIG. 7( b), a second scheme is a schemethat scans adjacent pixel groups by applying the preset horizontal scanrange x_scan_range having a size which is varied according to a verticaldistance between the effective pixel P and each pixel group to the eachpixel group, at least one portion of which is included in the verticalscan range y_scan_range preset from the effective pixel P. Referring toFIG. 7( a), 5 pixel is set as the horizontal scan range x_scan_range ona group 2 among the pixel groups included in the vertical scan rangey_scan_range, wherein a vertical distance from the effective pixel tothe group 2 is 3 pixel. 6 pixel is set as the horizontal scan rangex_scan_range on a group 3 among the pixel groups included in thevertical scan range y_scan_range, wherein a vertical distance from theeffective pixel to the group 3 is 1 pixel. Referring to FIG. 8( a), 3pixel is set as the horizontal scan range x_scan_range on a group 2among the pixel groups included in the vertical scan range y_scan_range,wherein a vertical distance from the effective pixel to the group 2 is 3pixel. 6 pixel is set as the horizontal scan range x_scan_range on agroup 3 among the pixel groups included in the vertical scan rangey_scan_range, wherein a vertical distance from the effective pixel tothe group 3 is 1 pixel.

As illustrated in FIG. 7( b), the scan scheme of FIG. 7( a) presets thehorizontal scan range x_scan_range according to a vertical distance froma pixel group to the effective pixel for a scan range to be a convexwindow W1 on a current effective pixel P. Likewise, as illustrated inFIG. 8( b), the scan scheme of FIG. 8( a) presets the horizontal scanrange x_scan_range according to a vertical distance from a pixel groupto the effective pixel for a scan range to be a concave window W2 on acurrent effective pixel P.

To set different horizontal scan ranges x_scan_range according to avertical distance from a pixel group to an effective pixel, a user candetermine an index that represents the vertical distance from the pixelgroup to the effective pixel, and can set a size of the horizontal scanrange x_scan_range according to the index as a look-up table.

The group information updating unit 14 updates information of a pixelgroup stored in the group information storing unit 11 according towhether there is the scanned adjacent pixel group and a pixel groupincluding an input pixel preceding the effective pixel in the operationsS15 to S17.

Specifically, when the scan result of the operation S15 shows that thereexists no group scanned by the group scanning unit 13, the groupinformation updating unit 14 determines that a current effective pixelis the first pixel forming a new pixel group, and stores groupinformation of the new pixel group including the current effective pixelin the group information storing unit 11 in the operation S17.

When the scan result of the operation S15 shows that there exists groupscanned by the group scanning unit 13, the group information updatingunit 14 can update and correct group information pre-stored in the groupinformation storing unit 11 according to the number of the scannedgroups and a group including an input pixel preceding the currenteffective pixel in the operation S16.

FIG. 9 is a flowchart illustrating a method for updating groupinformation pre-stored in the group information storing unit 11 when anadjacent pixel group is scanned according to an embodiment of thepresent invention. Referring to FIG. 9, the number of adjacent pixelgroups scanned by the group scanning unit 13 is one in operation S91.When a pixel group including a pixel preceding a current effective pixelis different from the scanned one pixel group in operation S91, thescanned one adjacent pixel group is merged into the pixel groupincluding the preceding input pixel, information of the scanned oneadjacent pixel group is deleted from the group information storing unit11, and information of the pixel group including the preceding inputpixel is updated for the effective pixel to be included in the pixelgroup including the preceding input pixel in operation S93.

Moreover, when the number of adjacent pixel groups scanned by the groupscanning unit 13 is one in the operation S91 and the scanned oneadjacent pixel group is the same as the pixel group including thepreceding input pixel in the operation S92, information of the pixelgroup including the preceding input pixel is updated for the effectivepixel to be included in the pixel group including the preceding inputpixel in operation S94.

When the number of adjacent pixel groups scanned by the group scanningunit 13 is two in the operation S91 and one pixel group of the scannedtwo adjacent pixel groups is the same as the pixel group including thepreceding input pixel in the operation S95, a pixel group which isdifferent from the pixel group including the preceding input pixel amongthe scanned two adjacent pixel groups is merged into the pixel groupincluding the preceding input pixel, information of the merged pixelgroup is deleted from the group information storing unit 11, andinformation of the pixel group including the preceding input pixel isupdated for the effective pixel to be included in the pixel groupincluding the preceding input pixel in operation S96.

When the number of adjacent pixel groups scanned by the group scanningunit 13 is two and all the scanned two adjacent pixel groups aredifferent from the pixel group including the preceding input pixel inthe operation S95, the scanned two adjacent pixel groups are merged intothe pixel group including the preceding input pixel, group informationof the merged two pixel groups is deleted from the group informationstoring unit 11, and information of the pixel group including thepreceding input pixel is updated for the effective pixel to be includedin the pixel group including the preceding input pixel in operation S97.

Finally, the group feature determining unit 15 determines whether acorresponding pixel group is effective according to the number of pixelsincluded in a pixel group determined as the update completion of groupinformation among the pixel groups stored in the group informationstoring unit 14 and the width-to-height ratio of the pixel groupdetermined as the update completion of the group information, anddeleting group information of a pixel group determined as an ineffectivepixel group from the group information storing unit 11 in the operationS18.

That is, the group feature determining unit 15 checks the flag grp_Flagrepresenting whether there is in an activated state about that the groupinformation update of the pixel group is completed in group informationstored in the register form in the group information storing unit 11,and checks the number of pixels included in a corresponding pixel groupand the width-to-height ratio of the corresponding pixel group on adeactivated pixel group according to a result of the check. When thenumber of pixels of a deactivated pixel group is less than the number ofpreset reference pixels, the group feature determining unit 15determines that the deactivated pixel group is different from a pixelgroup representing a desired pattern, and deletes a corresponding groupinformation from the group information storing unit 11. Moreover, thegroup feature determining unit 15 calculates the width-to-height ratioof the deactivated pixel group using the horizontal minimum value,horizontal maximum value, vertical minimum value and vertical maximumvalue of the deactivated pixel group. When the calculated rate is notwithin a preset rate range, the group feature determining unit 15determines that the deactivated pixel group is different from a pixelgroup representing a desired pattern, and deletes a corresponding groupinformation from the group information storing unit 11. For example, asillustrated in FIG. 10, in a case where there exist a plurality ofdeactivated pixel groups stored in the group information storing unit 11and the segmentation method of the present invention is applied to asystem that should detect the shapes of eyes for detecting red-eye, Thatis, the group feature determining unit 15 determines that groups 5 and6, which have many differences with the width-to-height ratio of eyesamong pixel groups illustrated in FIG. 10, are different from a pixelgroup having a desired pattern, and deletes group information ofcorresponding pixel groups from the group information storing unit 11.On the other hand, the group feature determining unit 15 determines thatgroups 1 to 4 of FIG. 10 have a suitable row-column rate, and outputscorresponding pixel groups to be used for a succeeding patterndetection.

As described above, an embodiment of the present invention determinessegmentation per progressive input pixel, and performs segmentation ofan image through a simple operation of comparing a pixel with presetvalues instead of a complicated operation, thereby quickly performingsegmentation of an image in real time. Furthermore, since an embodimentof the present invention requires no a microprocessor for a complicatedoperation, it can save both the time and the cost.

Hereinafter, an apparatus and method for removing red-eye in 2D imageusing the area information of a pixel group determined by theabove-described segmentation apparatus and method will be described indetail.

FIG. 11 is a block diagram of an apparatus for removing red-eye in 2Dimage according to an embodiment of the present invention.

Referring to FIG. 11, the apparatus 100 for removing red-eye in 2D imageaccording to an embodiment of the present invention includes a referencelength calculating unit 110, a pixel length calculating unit 120, asection determining unit 130, and an image manipulating unit 140.

The reference length calculating unit 110 receives coordinateinformation of an area including red-eye in an entire image, anddetermines the center coordinates of the area and a reference lengthfrom the center coordinates.

The pixel length calculating unit 120 sequentially receives each of thepixels of the image, and calculates a pixel length being a distancebetween the pixel and the center of the area.

The section determining unit 130 compares the reference length with apixel length and selects a pixel, the pixel length of which is less thanthe reference length. The section determining unit 130 determines asection including the selected pixel among predetermined sectionsaccording to a distance from the center coordinates.

The image manipulating unit 140 sets a brightness gain and color gainfor each section, and applies the brightness gain and the color gain tocorrect a pixel value of the selected pixel.

FIG. 12 is a block diagram of a method for removing red-eye in 2D imageaccording to an embodiment of the present invention.

Referring to FIG. 12, the method for removing red-eye according to anembodiment of the present invention includes operations S100, S110,S120, S130 and S140. The operation S100 receives the coordinateinformation of an area including red-eye in an entire image. Theoperation S110 determines the center coordinates of the received areaand a reference length from the center coordinates. The operation S120sequentially receives each of the pixels of the image, and calculates apixel length being a distance between the each pixel and the center ofthe area. The operation S130 compares the reference length with thepixel length and selects a pixel, the pixel length of which is less thanthe reference length. The operation S130 determines a section includingthe selected pixel among predetermined sections according to a distancefrom the center coordinates. The operation S140 sets a brightness gainand color gain for each section, and applies the brightness gain and thecolor gain to correct a pixel value of the selected pixel.

FIG. 13 is a flowchart illustrating in more detail the operation S140 ofcorrecting the pixel value of FIG. 12.

Referring to FIG. 13, the operation S140 of correcting the pixel valuemay include operations S141 to S143. The operation S141 differently setsthe brightness gain for each section according to the brightness valueof the selected pixel. The operation S142 differently sets the colorgain for each section according to a difference between a color value ofthe selected pixel and a target color value to be corrected. Theoperation S143 corrects the Y, Cb and Cr values of the selected pixel asexpressed in Equations 1 to 3 below.

Y′=Ly×Y   (1)

Cb′=Cb+(Sb−Cb)×Jb   (2)

Cr′=Cr+(Sr−Cr)×Jr   (3)

In the Equations 1 to 3, Y represents the Y value of the selected pixel,Cb represents the Cb value of the selected pixel, Cr represents the Crvalue of the selected pixel, Ly represents a Y gain for each section, Sbrepresents a target Cb value to be corrected, Jb represents a Cb gainfor each section, Sr represents a target Cr value to be corrected, Jrrepresents a Cr gain for each section, Y′ represents the corrected Yvalue of the selected pixel, Cb′ represents the corrected Cb value ofthe selected pixel, and Cr′ represents the corrected Cr value of theselected pixel.

Hereinafter, the apparatus and method for removing red-eye in 2D imageaccording to an embodiment of the present invention will be described indetail with reference to the accompanying drawings.

Referring to FIGS. 11 and 12, the method for removing red-eye accordingto an embodiment of the present invention starts from the operation S100of inputting area information to the reference length calculating unit110. The area information is an area including the segmented pixels thatare to segment pixels expected to represent red-eye in an entire imageintended to remove red-eye. The area is represented as the minimumhorizontal coordinate, maximum horizontal coordinate, minimum verticalcoordinate and maximum vertical coordinate of the segmented pixels.Accordingly, the area information input to the reference lengthcalculating unit 100 is the minimum horizontal coordinate, maximumhorizontal coordinate, minimum vertical coordinate and maximum verticalcoordinate of the area that is to segment pixels representing red-eye.The technology associated with the segmentation of pixels has beendescribed in detail through the 2D image segmentation apparatus andmethod according to embodiments of the present invention. Briefly, theapparatus and method for removing red-eye in 2D image according anembodiment of the present invention perform segmentation on an areawhere red-eye occurs by applying the image segmentation method accordingto an embodiment of the present invention, and input the coordinateinformation of a determined pixel group to the reference lengthcalculating unit 110, thereby starting segmentation.

FIG. 14 is a conceptual view of the area. As illustrated in FIG. 14, thearea is represented as an area between the minimum horizontal coordinatex_min, maximum horizontal coordinate x_max, minimum vertical coordinatey_min and maximum vertical coordinate y_max of the segmented pixels.

The reference length calculating unit 110 determines a reference lengthlen_max to be applied to the input area using information of the inputarea in the operation S110. To determine the reference length, thereference length calculating unit 110 calculates center coordinatesX_(—)0 and Y_(—)0 of the minimum horizontal coordinate x_min, maximumhorizontal coordinate x_max, minimum vertical coordinate y_min andmaximum vertical coordinate y_max of the area. Herein, a middlecoordinate X_(—)0 of the minimum horizontal coordinate x_min and themaximum horizontal coordinate x_max of the area is the horizontalcoordinate of the center coordinates X_(—)0 and Y_(—)0, and a middlecoordinate Y_(—)0 of the minimum vertical coordinate y_min and themaximum vertical coordinate y_max of the area is the vertical coordinateof the center coordinates X_(—)0 and Y_(—)0. Subsequently, the referencelength calculating unit 110 determines an average value ((a+b)/2) ofhalf (a=(x_max−x_min)/2) of a distance between the minimum horizontalcoordinate and maximum horizontal coordinate of the area and half(b=(y_max−y_min)/2) of a distance between the minimum verticalcoordinates and maximum vertical coordinates of the area as thereference length len_max.

The pixel length calculating unit 120 sequentially receives each of thepixels of the image in which an area is set, and calculates a pixellength being a distance between the pixel and the center coordinates ofthe area in the operation S120. Referring to FIG. 4, assuming thatcoordinates x_pos and y_pos are the coordinates of an input pixel, thepixel length len_xy is the square root of “(x_con)2+(y_con)2”. Herein,x_con is a horizontal distance between an input pixel and the centercoordinates of the area, and y_con is a vertical distance between aninput pixel and the center coordinates of the area.

The section determining unit 130 compares the reference length len_maxwith a pixel length len_xy and selects a pixel, the pixel length len_xyof which is less than the reference length len_max. The sectiondetermining unit 130 determines a section including the selected pixelamong predetermined sections according to a distance from the centercoordinates X_(—)0 and Y_(—)0.

That is, as illustrated in FIG. 14, the section determining unit 130selects a pixel included in a reference circle, wherein the center ofthe reference circle is the center coordinates X_(—)0 and Y_(—)0 and theradius of the reference circle is the reference length len_max.

The section determining unit 130 may apply a change rate of FIG. 15 inorder for color or brightness to smoothly be changed according to therelationship between the reference length len_max and the pixel lengthlen_xy, i.e., a position of a pixel included in the reference circle.The change rate is a concept corresponding to a brightness gain and acolor gain to be described below, and it can be understood as that coloris corrected by applying a gain which is gradually changed according toa position of a pixel for a natural color change.

As illustrated in FIG. 15, it is preferable to apply a change rate whichis continuously changed according to a position of an input pixel, butan operation can very be complicated for implementing the change rate inhardware or software. Accordingly, as illustrated as dotted lines inFIGS. 16 to 19, an embodiment of the present invention may select amethod that sets a plurality of sections according to a distance fromcenter coordinates and applies the same gain to one section.

The section determining unit 130 sets a plurality of sections accordingto a distance from the center coordinates and determines which sectionof the sections an input pixel is in.

The image manipulating unit 140 sets a brightness gain and color gainfor each section, and applies the brightness gain and the color gain tocorrect a pixel value of the selected pixel in the operation S140.Particularly, the image manipulating unit 140 applies segmented gainsaccording to the brightness and color of an input pixel in theoperations S141 and S142.

FIGS. 16 and 17 are graphs illustrating a change of a brightness gain inaccordance with a position of a selected pixel in an embodiment of thepresent invention. In the graphs illustrated in FIGS. 16 and 17, the xaxis represents a distance from a center coordinate (set the centercoordinate as 1), and the y axis represents the brightness gain. Forexample, in a case where an input pixel has a high brightness value, ahigh gain is applied to a pixel disposed in the outermost area asillustrated in FIG. 16. On the other hand, in a case where an inputpixel has a relatively low brightness value, a relatively low gain isapplied to the pixel disposed in the outermost area as illustrated inFIG. 17, and thus a difference with a pixel near the center coordinatecan be reduced.

FIGS. 18 and 19 are graphs illustrating a change of a color gainaccording to a position of a selected pixel in an embodiment of thepresent invention. In the graphs illustrated in FIGS. 18 and 19, the xaxis represents a distance from a center coordinate (set the centercoordinate as 1), and the y axis represents the color gain. As opposedto the above-described brightness gain, the color gain is set to have ahigh gain as a pixel is nearest to the center coordinate. However,similarly to the above-described brightness gain, in a case where acolor difference between a target color to be changed and a currentinput pixel is great, a high gain is applied to a pixel disposed in theoutermost area as illustrated in FIG. 18. On the other hand, in a casewhere the color difference between the target color and the currentinput pixel is less, a relatively low gain is applied to the pixeldisposed in the outermost area as illustrated in FIG. 19, and thus adifference with a center area can be reduced.

A brightness reference for selecting one brightness gain to be appliedamong the brightness gains of FIGS. 16 and 17 and a color reference forselecting one color gain to be applied among the color gains of FIGS. 18and 19 may suitably be selected by a user.

Moreover, in a case where the color of an input pixel is the same as theskin color or is a very bright color such as the highlight, the imagemanipulating unit 140 bypasses a corresponding pixel without correctingthe brightness and color of the corresponding pixel.

In a case where an image representing eyes becomes highlight having nocolor due to the reflection of light occurring in the center area of theeyes, the image represents more excellent natural eyes in terms of afine view. The image manipulating unit 140 may determine the brightnessgain in order for a section including the center coordinates amongsections including a pixel to almost be highlight by applying such anadvantage. That is, as illustrated as dotted lines in FIGS. 20 and 21,the image manipulating unit 140 may very brightly represent an eye imageby applying a gain nearest to 1 to a center section including the centercoordinates. In a case where such a gain is applied, as illustrated inFIG. 22, a natural image of an eye can be obtained, in which the corneadarkens toward and then sparkles at the center.

For example, in a case where a gain of a specific section determinedaccording to the above-described processes is determined in a YCbCrcolor space, the brightness value Y, color value Cb and color value Crof a pixel is determined as expressed in Equations 1 to 3 below.

Y′=Ly×Y   (1)

Cb′=Cb+(Sb−Cb)×Jb   (2)

Cr′=Cr+(Sr−Cr)×Jr   (3)

In the Equations 1 to 3, Y represents the Y value of the selected pixel,Cb represents the Cb value of the selected pixel, Cr represents the Crvalue of the selected pixel, Ly represents a Y gain for each section, Sbrepresents a target Cb value to be corrected, Jb represents a Cb gainfor each section, Sr represents a target Cr value to be corrected, Jrrepresents a Cr gain for each section, Y′ represents the corrected Yvalue of the selected pixel, Cb′ represents the corrected Cb value ofthe selected pixel, and Cr′ represents the corrected Cr value of theselected pixel.

As described above, embodiments of the present invention segmentsections according to a distance from the center of eyes and apply again which is gradually changed according to the sections, and thus canobtain a natural eye image after removing red-eye. Moreover, sinceembodiments of the present invention can remove red-eye in real time ina pixel order of an input image, they can perform a quick process andrequire no separate many memory spaces.

Embodiments of the present invention can perform a real-timesegmentation on the pixels of a progressive input image, therebyconsiderably reducing a time taken in the segmentation of the 2D image.

Embodiments of the present invention can omit additional hardwareresources such as a microprocessor by requiring no complicated operationfor the segmentation of the 2D image, thereby reducing the cost spentfor the implementation of hardware.

Embodiments of the present invention have no image-manipulated tracebecause an image after the removal of red-eye is very natural and canperform a real-time manipulation by a pixel unit of an input image,thereby reducing a memory space and an amount of operation for a red-eyeremoving operation.

While the present invention has been shown and described in connectionwith the exemplary embodiments, it will be apparent to those skilled inthe art that modifications and variations can be made without departingfrom the spirit and scope of the invention as defined by the appendedclaims.

1. An apparatus for removing red-eye in a two-dimensional (2D) image, comprising: a reference length calculating unit receiving coordinate information of an area comprising red-eye in an entire image, and determining center coordinates of the area and a reference length from the center coordinates; a pixel length calculating unit sequentially receiving each of pixels of the image, and calculating a pixel length being a distance between the each pixel and the center of the area; a section determining unit comparing the reference length with a pixel length, selecting a pixel, the pixel length of which is less than the reference length, and determining a section comprising the selected pixel among predetermined sections according to a distance from the center coordinates; and an image manipulating unit setting a brightness gain and color gain for each section, and applying the brightness gain and the color gain to correct a pixel value of the selected pixel.
 2. The apparatus of claim 1, wherein the reference length calculating unit calculates the center coordinates, and determines an average value of half of a distance between the minimum horizontal coordinates and maximum horizontal coordinates of the area and half of a distance between the minimum vertical coordinates and maximum vertical coordinates of the area as the reference length, wherein a middle coordinate of a minimum horizontal coordinate and maximum horizontal coordinate of the area is a horizontal coordinate of the center coordinates, and a middle coordinate of a minimum vertical coordinate and maximum vertical coordinate of the area is a vertical coordinate of the center coordinates.
 3. The apparatus of claim 1, wherein the image manipulating unit differently sets the brightness gain for the each section according to the brightness value of the selected pixel, and differently sets the color gain for the each section according to a difference between the color value of the selected pixel and a target color value to be corrected.
 4. The apparatus of claim 3, wherein when the selected pixel is highlight or has the same color as a skin color, the image manipulating unit bypasses the selected pixel without correcting a brightness and color of the selected pixel.
 5. The apparatus of claim 1, wherein the image manipulating unit determine the brightness gain in order for a section comprising the center coordinates among the sections to be highlight.
 6. The apparatus of claim 1, wherein the image manipulating unit corrects Y, Cb and Cr values of the selected pixel as expressed in Equations below: Y′=Ly×Y; Cb′=Cb+(Sb−Cb)×Jb; and Cr′=Cr+(Sr−Cr)×Jr, where Y represents the Y value of the selected pixel, Cb represents the Cb value of the selected pixel, Cr represents the Cr value of the selected pixel, Ly represents a Y gain for the each section, Sb represents a target Cb value to be corrected, Jb represents a Cb gain for each section, Sr represents a target Cr value to be corrected, Jr represents a Cr gain for the each section, Y′ represents a corrected Y value of the selected pixel, Cb′ represents a corrected Cb value of the selected pixel, and Cr′ represents a corrected Cr value of the selected pixel.
 7. A method for removing red-eye in a two-dimensional (2D) image, comprising: receiving coordinate information of an area comprising red-eye in an entire image; determining a reference length from the center coordinates and center coordinates of the received area; sequentially receiving each of pixels of the image, and calculating a pixel length being a distance between the each pixel and the center of the area; comparing the reference length with a pixel length, selecting a pixel, the pixel length of which is less than the reference length, and determining a section comprising the selected pixel among predetermined sections according to a distance from the center coordinates; and setting a brightness gain and color gain for each section, and applying the brightness gain and the color gain to correct a pixel value of the selected pixel.
 8. The method of claim 7, wherein the determining of the reference length comprises: calculating the center coordinates; and determining an average value of half of a distance between the minimum horizontal coordinates and maximum horizontal coordinates of the area and half of a distance between the minimum vertical coordinates and maximum vertical coordinates of the area as the reference length, wherein a middle coordinate of a minimum horizontal coordinate and maximum horizontal coordinate of the area is a horizontal coordinate of the center coordinates, and a middle coordinate of a minimum vertical coordinate and maximum vertical coordinate of the area is a vertical coordinate of the center coordinates.
 9. The method of claim 7, wherein the setting of the gains comprises: differently setting the brightness gain for the each section according to the brightness value of the selected pixel; and differently setting the color gain for the each section according to a difference between the color value of the selected pixel and a target color value to be corrected.
 10. The method of claim 9, wherein the setting of the gains comprises bypassing the selected pixel without correcting a brightness and color of the selected pixel when the selected pixel is highlight or has the same color as a skin color.
 11. The method of claim 7, wherein the setting of the gains comprises determining the brightness gain in order for a section comprising the center coordinates among the sections to be highlight.
 12. The method of claim 7, wherein the setting of the gains comprises correcting Y, Cb and Cr values of the selected pixel as expressed in equations below: Y′=Ly×Y; Cb′=Cb+(Sb−Cb)×Jb; and Cr′=Cr+(Sr−Cr)×Jr, where Y represents the Y value of the selected pixel, Cb represents the Cb value of the selected pixel, Cr represents the Cr value of the selected pixel, Ly represents a Y gain for the each section, Sb represents a target Cb value to be corrected, Jb represents a Cb gain for each section, Sr represents a target Cr value to be corrected, Jr represents a Cr gain for the each section, Y′ represents a corrected Y value of the selected pixel, Cb′ represents a corrected Cb value of the selected pixel, and Cr′ represents a corrected Cr value of the selected pixel. 